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©N-gram (N(±2JW±<DS^&T-*-5. ) 

smssbjm-ssb 1 omwm&t. 

JbESg 2 ©«»^8lc J:o T**£ttfc»*Ma]»<J!>W*i 
co^H JiJWiSBn©#i§^J©tB^iiig:i« ssa*t#s»^ig 
¥f§C0N-g r amWS^fii^^-g-trScffWH^^ 

©¥§§©N-g r amWg&Jg^lcSo'UT. H)f^co¥ 
r amcDM^SS^Sr-g-tJgEff-WmiS^r^U^^fiS-r-SM 

4 ©«*^a*«*fcc <fc.s#*£-r*nijR* i x« 2 

EKWiffifrtomSi^^l/SiJfc&fi. 

*BB#R£BAfc*J»BB»Bl::4st»T. 
±E#/&i8B3=&tt. IS;RJgl. 2X«3ESc©8cff£Kj 

B. 

fctfJ^BBBBCiSHT. 

±E**B»¥Bfi, 2 Xtt 3 E«©«S«-W 

SSi^E x^jfcjfcBBK: «fc o T£fiJc S *l£8Efrl$miHE 
;HS*SB LT. |*7*$S!l*<« L < n^wi&si^- 

»«iT*»j»B«BB. 
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[0 0 0 1] 

- * ics-3*t ^T^tf Btst ^ju £ ±fig-r Sctf wm§§ 
Aasn*fs**j**©ej*B^s**Ba-$-**j»B 

[0 0 0 2] 

&m&mft&tzsb\znm : E ; T)i<$:m^zj3mAm2c2ri 

^ent^ffg^fMLTN-gr am (N — 2^5 
A ; CCT, Nf42JW±Wg^g:T*-5. ) d 

«StLi@©*iS?!|w 1 i=w 1 , w,, -, Wl 

[0 0 0 3] 
[&1] 

L 

p (w! 1 ) = n p (w, Iwui-,- 1 ) 
t = l 

[0 0 0 4] w.teJgBjajw,'-©?^ t#g© 

&t>T. ±ERlfc*5t>T, BWP (w, 1 

TJsI&IC, ffi^P (A I B) «#§§X«¥l§JiJB^^ 

ftitc*5tt-5 rnj « t = itS^lstw^p (w, 

[0 0 0 5] tu5T. i5¥. iEttttWIBtf^N 
-g r am£«fr»T««#*BB©teB«l*]±Stt*^ 

&t>mA,\zmmistiT^z> («Atf. se^s^^i 

l"L. R. Bahl et al. . "A Maxim 
um LikelihoodApproach to 
Continuous Speech Recogni 
t ion", IEEE Transactions o 
n PatternAnalysis and Mac 
hine Intelligence, pp. 179 — 
1 9 0. 1 9 8 3^j RZf&&&ffiX1&2 rfii*^, 

«*£ttffi»fc. SP95-88, pp. 49-54. 

[0 0 0 6] L^U/«C*<6. N-gramlW7^-J 

mmmzttVTbm&mmzs-z.zw-mito^m m*. 
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«*&tKXiR3 TF. J e 1 i n e k e t a 
1 . , "Interpolated estfmati 
on of Markov Source Param 
eters from Sparse Data", P 
roceedings of Workshop Pa 
ttern Recognition in Prac 
tice, pp. 381-387, 1 9 8 0 , 

rs. M. Ka t z e t a 1 . , "Es 
timation of Probabilities 
from Sparse Data for the 
Language model Component 
of a Speech Recognizer", 
IEEE Transactions on Acou 
sties. Speech, and SignalPr 
ocessing, pp. 400 — 401, 1987 

mr5 <N-gr a mfl-R^^O B a c k - o f f ¥*t 
ft" . SP95-93, p 

pl-6, ¥^7^j #ra 0 ) ^ ^7XM "sj^fi 
N-g r amS/t7/-^ORS8'>St5#S («i 
«, S6*ft«Xtt6 rp. F. Brown et a 
1. , "Class— Based n — gram mo 
dels of natural language", 
Computational Linguistic 
s, Vol. 18, No. 4, pp. 4 6 7-4 7 9, 
1 9 9 2$j , ftffc&ffiXiSS 7 TT. R. N i e s 1 e 
r et a 1 . . "A Variable-Leng 
th Category — Based N— gram 
Language Model", Proceedin 
gs of ICASSP ' 96, Vol. 1; p 
p. 1 6 4- 1 6 7. 1 9 9 6fj St/«gffiXi8 

K^^r . n?m mmm^&mm^, s p 9 5 - 7 

3, pp. 1-6, ¥J«7^j #JH. ) «Wft£<«3R 
[0 0 0 7] mtE* fkmiWzfottTWVttfVrtltlT^Z 
©*B^--***«>aa>fctH»T**. 0#§gco 
Z>o Lft*Lttft<&. ^-:?fi£*f ^T^IC, m<0?X 



[0008] emsoHBSHPftts^atLT, mi§ 

Ot*— *Tffr£LfcN-g r ami, IM^X^Or- 
^TMLfcN-g r am££fi^#JCJ:Drg£T<5 

^mmWtXMk. 1-3-19, pp. 37-38, ¥ 
& 8 ^ 9 £ j ft*&ffi£iSc 1 0 . m*&t>\ "* 

^BWK^wsttwwsB^^uoa^oas*" , b* 

^S^IJIStf&Xm, l-Q-2 4, pp. 15 7-1 
5 8, ¥sE6^3/3j #Sa o ) jstfts. 
[0 0 0 9] WAtf, t¥*a«X«9<D^tt*ffl^fcffi 

(a) /Mf^^h^, fi^tf^-C^S^X HcflD*. 
l^RSSwiTSi, /h«T-*X h^TmlsimS 

B£H»K«;b*i-5 

(c) ^4-X h*>SJKir*ffi9. N-gramtrM 
[0010] 

Lfrmm lt^^ot, wr^e^hc* 

[0 0 11] *38W©Bfl9tt«±0)WB.«*«f*L. ft 

bt« z tare* «**rBM&B££0t-r« zh\z$> 

[0 0 12] 

B^ttttWSB^xJl^tiESWtt. RJf^oJt&o^x^ 

Tft*^*«l:»H©N-gram (Nte 2 £Lfc©e$* 

) coS^ffi^^^gT^^lOiS^^Sch, 
±E» 1 <Di%W¥®lZJ:vTmW2ftrz&*7>&m<DM 
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vft&ttrc&ttsBBttBQjpB^jonirooBBftoui 

[0 0 13] lll«q|2Ett«!)|«tt-mm%7 : ;^ 
fifc^Bte. W^qf l E«©&frWWB ; E7 r ;i'£j6£gfil;: 

CO 0 1 4] SSC, IIA9l3ettOttttA9«C%^ 

X^fi£D^|§05N-g r anKnm&v&m&stt&mffj-g 
its. 

CO 0 1 5] *£9ilCtt£B$a4Ett®**Bft£B 

S«*.fcBJ»B*BBlc*fr»T. ±EB*KBPBtt. 
Mf*JS 1 . 2XIJ3 E«OiK^e«JS-S^:5r;u^^gBJc 

[0 0 16] 4:H9HCff«H4»9f se«0*)irH«|ttH 

***S:©BBfiK*tttti L3t«Sfrjrr* C i C «fc 

■««K*t»T. ±E*^rBN^ftll. 11:09(1 ; 2X 
»4 3E*©«5ttfl9SHqE5F;U<feriE«|licJ:-aT*dESn 

*H£S*B*Snfc«*K©3ii©BfeilSt*jt**# 

&^£rr 5 Z. t6»iit*. 
[0 0 17] 

#®A7^-^lcSt?^T±E^^l : ^JCC0*ig{giJi^ 



«*J»BBBfiC*t»T, #8Bg£8B4 SAy 77^ 

StifcttlrWBB^xJM^'J («T« tttrttBS^Ey 
Wt'Jilr^. ) 2 2rt«*x*B£{b$ft*:j&Efr6<) 
WB'ExJU&IMSLT. SKBBO^B*B|||««(C. 

«a 3 «fc 5 (cBB«K®ig 0 a**ff 3 ¥i§{Ri«fe? 

C0 0 18] CCTBt»*tMMW«*5 s *tt, 
timi§^e)^-5#^x^ni©T-^7. h^-^ 3 0 -m 
(m=l. 2, -, M) £K#a*lTB*&¥B«T-* 

2F*lJfcfc©T*oT. «i»rfl$rtrB*7\;P2 2tt. 

ram (N=2. 3) *Btr. *XBICfll«|IMiBBtC 
*<r>Ttt. SKScffi-WBli^x^^^fig-r-St^fc. M 
A P (Maximum A- posteriori P 

r o b a b i 1 i t y : B*9£ff¥) Mfe&&m^jt 

Wti'Xi'Wf+X 1-^—^3 0- 17!>M3 0 - 
MIC*^JT^fi£Sn-5N-g r am£^tfSSfrl$s!§ 
^7*^3 1 £^iitrftf$£-r-3— ^ §WO#£^Xl' i 
Ot^X h^-* 3 0 - i eV&*oM£ LTMA Pfcjfc 

itfttffofcB. Bac k-o f fsprtrts&Ki.Sipifi- 
ft &g £ ^fr IT <5 c: <h K =k 0 * x BJfcfl: 3 tlfc N - g 

•5. dCT. BWO^^X^ i O^+X h^r— ^3 0 
-itt. ^FfS^x^O^X 1-^-^3 0 -m (m = 
1, 2, M) <Dot>(D l-DV&'fefXZtDTlrZ h 

[0019] sr. MAPm-SL&iz^zm&mmvwitii 
tz-D^xm^. ®fir. N-gr am<»m&mmz, u 

L (Maximum Likelihood; m.X) & 
f£8HzJ:0niii2iT.2>. «5t«jfcjfctt, BSJLfc-y->^ 
;HS CT«:to% v ^X r-x-^CD^B) xlC^fLT. 
a»**D****-afci*CBBxta:*3tBH» f 

(x I p) SrS^tc^ti-^MtLT. N-gr amli 

[0 0 2 0] 
CS2] 

P«t = a r gmax f (x I p) 
P 

[0 0 2 1] ZZ\X. USargmaxlJ. pt»bL 

fc<t#KW&©§i&a*B*£ftsi#©p©e£^-rM 

b i g r amcOS^fii^p (w, | w k ) tr>l>T#?t, 



(a) N : <¥HJH?*X H^-^OWw^&gtlH] 
ft. 

(b) n : h^-^tO^ffiWwfcW^ffl 
3i[E]ft. 

(c) p : i|i§£w k ;^£w l '\£>iII£&S^o 

[0 0 2 2] dO££;tgWft f (x I p) ^§§w 

nUtno*BK:jBt<0tt3^ (N-n) BT*6*6, * 
[0 0 2 3] 

[S3] f (p) = p° (1-p) *-» 
[0 0 2 4] f (p) <0«;fc{tftftd 1 o g f (p) / 
dp = 0 &m< Z.t\z£0. N-gramOIiSSJ 

[0 0 2 5] 

[R4] P ml = n / N 

[0 0 2 6] ^oT, t»LJ|iB9lw k w l 3»«|UH7 ff -^± 

J§££ttTL£?o cniC^LT, MAP (Maxim 
umA-pos ter i or i Probabi 1 i t 
y Ji«lC*5^TJ4, **ii*ffi*ffllr> 

x. ^iLfcit>7 p ;nax(c*fLT. w»nm p 
$n^r«?TOh (p i x) **^tr-s«tL 

X, N-g r amoa»«W^SC-C*ai>5h-5. 
[0 0 2 7] 
[ft 5] 

p m = argmax h (p I x) 
P 

[0 0 2 8] ::t, B a y e s fli]*fflt>* ,±E» 

[0 0 2 9] 
W6] 

Pi 4! = argmax f (x I p) g (p) 
P 

[0 0 3 0] :c*C, g (p) tt. ^«r«^X^05r* 
X hr"-^ 3 0-1 7iS3 0 -MlcS^X'f #ftj££ 

PtA? 

= a r gma x {p' (l-p) a 
P 

= a r gma x L (p) 
P 

[0034] m&L (p) a*§*£&*fc#<a 

*ttdlogL (p) /d p = 0£plc:^T#K<h, 
migcOb i g r am<D&&mmvw^^<»<£-ol~* 

[0 0 3 5] 

[&8] p w = (n + a-1) / (N+a + 0-2) 
[0 0 3 6] A7^-^aS^flH ^jffittfli 
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N-g r amOl»«*p0)»mMT*S. f 
MAPiSffi^ffl^t, N-gramW 
ff*tt«£*im*H8<t OmznZftlfig (p) KflEig 
ft£U CO*W»WfclMIK:«ai!Snfc*>y;Ht4: 

[0 0 3 1] &HX, b i g r amSrWfCi: 9, MAP 

t\ a»«*pCO*ffir»ffii:LT^-^»flf (ap«-> 
(l-p) CCt, aStfjSIS^- ^#ffi<Z)IE<0 

(a) ^-^»*tt2«»*©B«*a*W»*-Tr, M 

sic****, tt*. **«*»tfe(r^Ttt, 

[0 0 3 2] ±Eft60MAP*«»0«*ttJe5i:, 

a»«* PsiA ptt, *gHftf (p) <t¥Bn»^g 
(p) t*ffl(fiT*sc<Ocfc"5(c**e>n*. 

[0 0 3 3] 
[38:7] 



'ap-- 1 (l-p) 



sxaiKXiK5»Hi. ) . 

[0 0 3 7] 

(ft 9] w = a/ (a + fl) 
a 2 = (a/3) / { (a + 0) « (a + fl+1) ) 
[0 0 3 8] ZLtlt><Dj£$: a, a + 0 K^XffiK 
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[0 0 3 9] 

ISCI 0] a= [ u l il-u) ) /o*-u 
a + {u (1 - u) } /o 2 - 1 
[0 0 4 0] EU:<fc9. iir^hr-^^^S^ 

0«rJBfr>T. b i g r a m<Dm&mm*&&Z> C 

[0 0 4 1] Ctl*T<BH«&tt, ¥f§Ob i g r amlC 

•O^T<nfrV>mm~Z$>r>tz.tf. —mz. mapissi; 
<t<5N-gr am©I»«¥p (w B I w,»-«) tt, S 
ffiTO*IBw k 6il[tt^>WIB9lw | o-i&Bfti||A. /t^* 

(a) N : ^ffi^X K^-^^O^S^Jw^-'coaB 
giSfS (c (w,h) ) % -T&fo^, fflSMfto^Lggyij 

(b) n : h?— ^ + ©*H«lw,»Cl)HlS 
©S: (c (w,») ) , "Tttto*, MS*f^^#|§^iJoai 

A = 2c, (w a ) p, (w a 
i 

[ftl 2] 

i 

[0 0 4 4] CCt, c, (w,h) ifc:*5<,> 
T¥»^Jw,»-i<BHjS«ffT*9* Pj ( W| |w,H) 

(a) N : SW^jl^x^ i hr-^ 3 0 - 

(b) n : @f$CD#^X^ i (D^^rX h^—f 3 0- 

Z±m<D&7RZ*&9\zKX?ZZ.t\ZJ:r), MA Pit 
[0 0 4 5] B a c k-o f f ^FftYbfefZctS 

-**JHHTfc, tt«L*^*BM*«#aEL, MAP* 
£j££/B^Tt>* N-gram©I8IIW0tftoT 

P s (w D I w, B -0 
= Ph (w p I w, H ) , c, (w,«-«) > 



[0042] &\,*x\ MApmm&zm^fz*x?mfc 
immiz^xm^z. ±m<DUApm^&\z^^N- 

g r am%?7s#m&itlZfcm?Z>rztblZ. m3\z^T 

-*fr%ft&Znttftfe**$<OT*7s V"r-5 3 0 
-mfCl^N-gram^fgaiil, 

T. ^^7*ON-g r am^f gliaai^&Lt 
^7^l;^N-gram 

4MC*5»*N-g r amlg8$iiltl^g(:j;0* 
#<s>n<5o C<0*H9^S:^-^^ifi[^L, m4 \Z 

[0 0 4 3] 
[&11] 

I w/" 1 ) /2c, (wx 1 ' 1 ) 
i 

Wi 8-1 ) VSc, (w/- 1 ) 
i 

&z>o lzxDmmit. *ftw\z&2>27.i7mfe{tm 
m\t* ±T<Dm&mmz&iL\z&&z^mT'&z>rz#>. 
m&mm<Dm*i\ztiiZ>£\zm*>ts:^ 0 mm&j*sm*f 
\zmm-?z>mz, mm titta n—^v^^j- 
jT-wmTzmt* nzmmttsftT^m>£. jev^ 

a c k-O f i^-mtm fE*tt«:*itt4# 

fig. ) ^Mit, ctis^paiHswsirs^fe*^ 

[0 0 4 6] ^HyiJw^^^^x^COy-^ hf- 
* 3 0 -m«C&sns«'&tt, ±830^X>aiSft«lS 
(CJ:0, a^fig^ PstAP (wJw,h) ^^l- 
•J>y (Turing) it^feJCctO, «¥ P!ttf (w a 

^co^^X h^-^3 0-mcogS (c,-(w,-) ) £ 

w,W#^^X £ C05r4^X h^-^ 3 0 -mlC^ft 
fc<r>#«MIK:»LT. (n-1) -gram^ISI 

<fc*tlfcN-g r am05p»{b»oa»«*P s (w D 
I w,»-i) »4^cSCT«Sn*. 
[0 0 4 7] 
[S13] 
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= a(w,°->) P s (w D | w 2 Q -0, 

= PS (W„ I Wj"* 1 ) , C| (w, 0 ' 1 ) 

[0 0 4 8] ±ffi<B&l 2\Z&^T* PhtefXZmfc 

P h (w 0 I w, 0 " 1 ) 
= (c, (w,°) + 1 } / {c, (w,°) } 

x {n c , (w,») +1) / {n ci (w,») } 



c , (w,°-») = 0 , c j (w 2 D -0> 0 

= 0, c, (w 2 D -') = 0<£><h# 
[0 0 4 9] 
[S14] 



Pmap ( w i n ) 



[0 0 5 0] n e tt, ^^^X^7(7)f+Xh 

^-^3 0-m + «CcSfflSr*mB?iJ<oa*»[ (Rfc 
0) T&D, Sfc, S13T, a (w,»-») tejESfbO 

[0 0 5 1] 

[SI 5] a (w,»->) =Aa/Ab 

Aasl- 2 Ph (w, I w/' 1 ) 

w a : C, (Wi") >0 
Ab=l- 2 Ph (w B | Wi 0 * 1 ) 

w, : c, (w, 1 ) >0 

[0 0 5 2] £1±<DB ack-of fT»ftftt««L 

[0 0 5 3] B5H Hf^T^^g^ OC^oTH 

3 0 - m«Ett**Mf *X hf-* / * 'J 2 
e>****m(07^;* hf-^ 3 0 -m£Sfc*ai"r. # 
HT. 2lC*5<^T, 7.7* y 7 S 2 C*5(r>T. 

^^ttJLrcS-^X^OT-^X h^-^ 3 0 -mfcS^ 
Tg*it^ffi^ffll^T&^7,^S(C^I§C0b i g r am 

S 3lC*5HT, &9X>7U<DWmb i g r amc£>ig;f£fil 

¥^/x&tfj!JD«#ifc02£&l O&tf&l 1 SfflHTtf 

£ i Ot-^X hf-^ 3 0 - i <DV&%lSli<Drt7 * — * 

b i g r amCOS^ffi^P^rtt'Sf'r^p fit, X^iy 
5 IC&^T, 3-*X^«Oi}iBb i g r amOS8 

wpi:s^»t, b a c k-o f f ^mftmmzn'n 

U ffi3fg<0ft^X*«<&J|iBb i g r amOS&tt93 

^^U2 2icE*r*. ci±t, TO^E^iftjaa* 



[0054] jfcir^T, Huc^rmmisioi 

(a) (b) SlpJi&3>f^h^7 

X, (c) 5fcfrtt». KtfttttKMtO'J^h, (d) ffi 

aa**^*^/^*—*. (e) esai»«* 

[0 0 5 5] Hll:*^ B*©«F*^f4T-f ir □ 
g52fcA^^tl^o 49ttMttl8B2»« A*StlfcfM 

1 6*A^r^ h9A«ft«r*tT3 4*7CO»«/^5>C 

[0 0 5 6] ¥BiHdB4tt. «7>-/t^ • tT^ tf*4» 
flStefcJfl^T, Ay77^ : E , J3^LTA**ft^ 
*—&<D~J-'—*\zm-J^T* fiHMM 1 1 <h¥ 
§§Sfl 2ttffltf»TJ|lB«K*«fflU**ttH|CLT 

fca&tc, a^HMM l iatf*B*«i 2tl:i^T 

tf7l/-A§^) iit)l:Ay77^ ; &U5S^LT* 
B«I^i&«6l;:iH;&-r*. 
[0 0 5 7] mB<SRiK^a56«, *BBB^S54^6A 
y77«'J 5*^LTtB*Sn*¥BfiBl«zS^ 
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tzmm^mm^^^mrnvx. &7m$wmL<mtth 
vikm^mmmmz. ^pmmmmai^^mmmnmT 

*l§&Ift£&gi$6te. f?fWIL-±5tt4j'^j? 

[0 0 5 8] MZ.II. S2tC^-r«fc^ \Z. (i-l)# 
B»^l§W M 0^lc. Ir^Ja,. a,, . a>£& 
S ISSO^iiWjTiKSi^lC. ifi§§w M <z>m§§{Stft 
<tUT6^W^iKWa, Wb, Wc, Wd, We, Wf 

&w&Lxm>. cct, ntim3^<Dmmis.mwa, w 

b, Wc©i^ltt/ x /T*5i:L, ^#3^>0^ 
m&mv/d, We, Wf cD«^if^fi/y/T*S<»:-r 

fiStS!^^** "x/a/a/' T*-5±^e, 3 ~DCO 

mm&m.) e>o-£>mxm&mt>m^Us$i (mx.ii. 02 

ff-trzmmmmomm^mfrxxiztsiK yx$>z<nx. 

mm&mvim&gmmz 1 -z><D&-&m&mT. m 2 <dm 
xu. mmsm/x./izttLTi^><z>&m&mL, mm 
^M/y/\znvx 1 zxDum&m-?. 
co 0 5 9) zuKDnm&mtz&^Tit. mmmm<D^ 

ts3^xo^mmzfiiLx^.m$nx^^A^ *&Bmz\ 
tuzm*>?. ftftTzmm&mommgmt. mm^m 



[0060] &,±<DmmMmz&^x. ^mmm^2 

£/&gB2 Otte. MA.lt. TiSZJVnTmwmteiiom 
>¥z.-*X'ffil&t<tl. A'>77^tU3, 5<i:. 

hmm^^ij 1 1 t. mm&mt^'j 1 2i. 

*x b^-^^u 2 1 1. mmtQwim^TAx*: u 2 

2 tit. ma.h;\- F^-fx^^^euTifoigitgB-c 

[0061] si±mmMmizi5^xiz. mmm&&4t 
mm&m.%i&&6 tzm^x^pmrnzft^x^zfr. 

*&BJHtzftizm*>-r. MA.it. h^HMM 1 l£#ra 
-rz^mmS&i:. Wx,l£On e Pass DP7A 
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